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Abstract 



We show that the algorithm for computing an element of the Clarke gen- 
eralized Jacobian of a max-type function proposed by Zheng-da Huang and 
Guo-chun Ma in [8] can be extended to a much wider class of functions repre- 
sentablc as a difference of max-type functions. 

1 Introduction 

Clarke's generalized differentiation constructions are employed in a variety of nons- 
mooth optimization techniques. The relevant generalized subdifferential and Jaco- 
bian are arguably the most common tools applied to a wealth of essentially nons- 
mooth problems (see the classical works [H El El E]). The new applications em- 
ploying the Clarke Jacobian are still being developed; for example, a gradient bun- 
dle method essentially based on Clarke subgradients has recently proved successful 
in solving eigenvalue optimization problems [21 [3], and the nonsmooth Newton's 
Method [10] has been applied to important classes of nonsmooth problems, such as 
various nonlinear complementarity problems, stochastic optimization, semi-infinite 
programming, etc. For a brief but thorough overview of recent applications we refer 
the reader to [8]. 

Let a function F : IR n — > IR" 1 be locally Lipschitz around a point x £ IR n . By 
Dp C IR™ denote the set of points on which F is differentiable. For an x G IR n let 



where by Limsup we denote the outer set limit (see [H]), i.e. the union of all limits 
of all converging subsequences, co is the convex hull, and V-F(x) is the classical 
Jacobian of F at x G Dp. The set dF{x) C K nxm is the Clarke generalized 
Jacobian of F at x. For a locally Lipschitz function this set is always nonempty and 
bounded (see [1]). 
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Let a function F : IR n — > IR m be such that each of its components is a pointwise 
maximum of a finite number of smooth functions, i.e., F = (fx, f%-, ■ ■ ■ , fm), 

fi{x) = max fij(x), V« 6 I = {1, . . . ,m}, (2) 

where Ji, i £ I are finite index sets, and : IR n — > IR are continuously differentiable 
for all xeJR n . 

It is impossible to compute the generalized Jacobian of max-type (or a difference 
of max-type) function from only the first-order information at hand (i.e. from the 
gradients of the component functions). The best one can do is to use the estimates 
like the bounds in [6j for quasidifferentiable functions. The algorithm suggested in 
[8] is essentially an elegant simplification of the method from [7] ; both are motivated 
by the observation that for some important applications, such as Newton's method, 
the computation of the whole set is not required, we only need one arbitrary element 
that surely belongs to the generalized Jacobian. 

Because of the max-type structure of the function, for every given point and 
any direction there exists an adjacent open set on which the max-type function is 
smooth, and hence the limit of the relevant Jacobians belongs to dF. The job of the 
algorithm is to carefully select the relevant gradients to build an element from the 
generalized Jacobian. In addition, the direction in [8] is chosen in a way to minimize 
the computation cost. The sole goal of this paper is to demonstrate that the original 
algorithm can be applied to a wider class of functions; we do not discuss the issues 
of finite precision and complexity here: this has already been addressed in [8] in 
detail. 

We discuss the algorithm in Section [2] and prove its correctness in Section 



2 The Algorithm 

Let F : IR n — > M m be such that F = G — H, where both G and H are max-type 
functions, i.e. 

G = (gi,---,9m), H = (hi,...,h m ) (3) 

with 

gi(x) = max gij(x), hi(x) = max h ik (x), i £ I = {1, . . . , m}, (4) 
je.Ji keKi 

where gij, ho- : IR n — > IR are C 1 functions, and J-i and Ki are finite index sets for all 
i € /. We will also use the notation F = (/i, . . . , f m ) with fi = gi — hi, i E I. 
For an x 6 IR n and each i € / define the active index sets 

Ji (x) = I j | 9ij {x) = maxg ij (x) \ and K^x) = lk \ h iko (x) = maxh ik (x) \ . 

By V/ we denote the gradient of / : IR™ — > IR, and by e; we denote the l-th 
coordinate vector: (e^); = 1, (ef)j = 0, I ^ j. 

The Algorithm Al is an extension of Algorithm 2.1 in [8]. The basic idea is 
to consider the individual sub differentials of each of the functions gi,hi, i £ I and 
from each one to choose one vertex (gradient) in a way that all the selected ver- 
tices correspond to the same direction in which all the aforementioned functions 
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are differentiable. Subroutine SI does the selection per se, while Algorithm Al on 
Step 1 iterates through the functions gi,hi, i G / and calls to Subroutine SI on each 
iteration. On Step 2 of Algorithm Al the gradients selected on the previous step 
are used to build an element £ G dF(x). 

Algorithm Al 

Input: A point x G IR n , finite index sets Ji,Ki and functions gij, j G J« and hik, 

k e Ki, i e I. 

Step 1: Fori G I compute Tj(x) =Sl(sc, Ji, {gij}jeJi), Si(x) =Sl(x, K, {h ik } keKi ). 
Step 2: Compute 

£ = (VflijiCa;) + V/ii fcl (x), • • • , Vg mjm (x) + V/i mfcm (x)) T , 

where ji G Tj(x) and fej G are chosen arbitrarily for each i £ I. 

Subroutine SI 

Input: A point x G IR n , a finite index set J and functions <7j, j € J. 
Step 1': Compute the active index set 

J{x) = {jo G J'lffio^) = 

let T°(x) = J{x). 
Step 2': For / = 1,... ,n let 

T / (x) = |t GT'- 1 (x)|V 5it0 (x) T e; 

Output T n (x). 

In the next section we prove the following result. 

Theorem 1 If F = G — H, where G and H are defined by equations ([3])-(j4|), Algo- 
rithm Al is well defined, and £ generated by the algorithm is an element of dF(x). 



max gj(x)}, 



max Vg it (x) T ei 
t€T l ~ 1 (x) 



3 Proof of the correctness of Algorithm Al 

Our proof of Theorem Q] is essentially along the lines of the proof of Theorem 2.1 
in [8], albeit is a bit shorter. We need to introduce a few definitions and technical 
results first. 

Definition 2 A continuous mapping f : ]R n — > JR m is said to be PC 1 on an open 
set U C IR n ; if there exists a finite set of C 1 functions fj:U—> ]R m ; j G J (with 
|J| < oo), such that for every x G U, fix) = fj(x) for at least one index j G J. 

Recall that a directional derivative of a function / : ]R n — > 1R at a point x G !R n 
in the direction y is the quantity 

fit \ r f(x + ty) - f(x) 

/(x;y) = hm . (5) 

All PC 1 functions are directionally differentiable, i.e. the limit (0) exists for all 
directions y G TR n . The next result follows from the definition of the directional 
derivative. For a detailed discussion see [UJ Chapter I, Corollary 3.2]. 
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Lemma 3 Let f : IR n — > IR be a pointwise maximum of a finite number of smooth 
functions, i.e. for all x G IR ra 

f(x) = max/j(x), 

where fj : IR™ — > IR, j G J are continuously differentiable, and J is a finite index 
set. Then for every x G IR n the function f is directionally differentiable along an 
arbitrary direction y G IR", and 

f'(x;y)= max f'Ax;y) = max Vfj(x) T y, (6) 
jeJ(x) jeJ(x) 

where J(x) is the active index set: 

J ( x ) = {h G J I fj (x) = max fj(x)}. 

3&J 

The following result is proved in [9j Lemma 2] (also see [8]). 

Lemma 4 Let F : IR n — > JR m be a PC 1 function in a neighborhood of x 6 IR n ; then 

d[F'(x;-)}(0) C 8F(x), 

where [F'(x;-)] : IR n — > K m is the vector function of the directional derivatives of 
the components of F at the point x. 

We are now in the position to prove our main result. The proof essentially follows 
the ideas of the proof of Theorem 2.1 in [8]. 

Proof of Theorem □ For each i e L let Tf{x) := Ji(x), S°(x) := K{(x), and for 
each I G {1, . . . ,n} define the index subsets T-(x) and Sl(x) recursively 



T\{x) = {tGT i 



S l i (x) = \seS< i -\x) 



Vgit(x) T ei= min Vgij(x) T ei > ; (7) 
jer^ix) I 



Vh is (x) T ei = min Vh ik (x) T ei } . (8) 
kes^ix) 



It is not difficult to observe that the sets Ti(x) = T™(x) and Si(x) = S™(x) are 
precisely the sets obtained after the execution of Step 1 of Algorithm Al. Since 
the index sets Ji(x) and Ki(x) are nonempty and finite, the minimal values of the 
scalar products in ([7|) and §E§ are attained, and hence on every iteration of Step 2' 
of Subroutine SI we generate nonempty sets. This means that after the execution 
of Step 1 of Al we end up with nonempty finite sets T™{x) and Sf{x), i G /, so we 
can choose the corresponding indices ji, . . . , j m , k%, . . . , k m on Step 2. 
Let 

T :={V gij (x) - Vg it {x)\j G Ji(x) \Ti(x),t eTi(x),i£ 1} 

U {Vh ik {x) - Vh is (x) \keKi(x)\ Si(x),s G Si(x),i G /} . 

It follows from ([7])- ([8]) that all the first nonzero components of all elements in T are 
positive. 
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Let e denote a positive number smaller than the minimum value among the first 
nonzero components of all elements in V and let M be a positive number larger than 
the maximum value among the absolute values of all components of all elements in 
T, i.e. for each a = (0, . . . , 0, a k , a k+ i, . . . , a n ) T G T we have |a$| < M, i = k . . . , n 
and a k > £ > 0. 

Let 

V = (-Ai, -A 2 , • • • , -A n ) T , 
where A, > 0, -r^ < ttV, i = 1, . . . , n — 1. We have 

n 

\0Li 



a T y = -^A 

n 

< -AfcOfc + \\ui\ 

i=k+l 
n 

< -X k a k + M Xi 

i=k+l 

= -\ k a k + M\ k T (j^-r) 



< -X k a k ■ M (—) A/,. 

= A fc (e - a fc ) 

< 0, 

which means that a T y < for all a G T. Observe that the set 

U = {y \a T y < 0;Va € T} 
is an open convex cone, which is nonempty since y G U. Hence, we have 
(V 5ij (x) - Vg it (x)) T y < 0, Vj G J ! W\T ! (i),fG T;(x) 

and 



(VMs) " V^ t (x)) T y < 0,V& G i<Q(x) \ Si(x), s G 5 f (x) 



which implies 



max Vgu{x) y= max Vgjj(x) y (9) 
jeJt(x) teTi(x) 



and 

max Vhi S (x) T y = max Vhi k (x) T y (10) 

keKi(x) seSi(x) 

for all y G [7. It follows directly from (J7|) and (jHJ) that for every i G / we have 

Vg itl {x) = Vg it2 (x) Vti,t 2 G Ii(:r); V/tj Sl (x) = Vh is , 2 (x) Vsi,s 2 G Sj(x). (11) 
Hence, for each i G / we get 

/? (x; y) = V 5ij (x) T y - VMxfy = ^ (12) 
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for all y £ U, and any arbitrary choice of j £ Tj(x) and k E Si(x), i.e. f'(x;y) is 
linear in y on [/. 

We next show that £ £ dsF{x). Fix an arbitrary y £ U. Since £/ is an open 
cone, for all t > we have ty £ U together with a neighborhood; then from (|12p we 
have V[F'(x; -)](ty) = £, and hence 

£ = ]imV[F' (x;-)](ty) € d[F'{x; -)](0) CdF(x), (13) 

where the last inclusion follows from Lemma [3] and the observation that F is PC 1 . 
The proof is complete. 
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